Adaptation in Constant Utility Non-Stationary Environments

Environments that vary over time present a fundamental problem to adaptive systems. Although in the worst case there is no hope of eeective adaptation, some forms environmental variability do provide adaptive opportunities. We consider a broad class of non-stationary environments, those which combine a variable result function with an invariant utility function, and demonstrate via simulation that an adaptive strategy employing both evolution and learning can tolerate a much higher rate of environmental variation than an evolution-only strategy. We suggest that in many cases where stability has previously been assumed, the constant utility non-stationary environment may in fact be a more powerful viewpoint. An adaptive system within an environment performs two basic tasks. First, there is the search for, and the representation of, regularities in the history of interactions with the environment. Second, there is the attempt to gain some advantage from the constructed representation, by basing future actions on the assumption that those regularities will persist. If that fundamental adaptive assumption fails utterly, and the environment totally lacks such regularities, adaptation can provide no beneet.